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ABSTRACT 

An evaluation was conducted of the effectiveness of 
the Adult Educators Development Project (AEDP) at Lehman College of 
the City University of New York, a 3-year professional development 
program combining meetings, research projects, site visits, reading, 
and writing that was designed to facilitate the professional 
development of instructors in adult basic education, literacy, and 
English for speakers of other languages. Data for the evaluation were 
obtained frorn interviews of a total of 11 participants from the 
project*s 3 program years, limited observations of the program 
seminars, and a review of various project documents. According to the 
participants interviewed, the project objectives were initially 
unclear difficult to grasp, thus fostering a sense of disorientation 
and confusion. Eventually, a process of empirical inquiry was 
initiated at the project sites, and the AEDP*s goals were brought 
into clearer focus. Most literacy teachers and administrators who 
participated in the AEDP praised it as a vehicle for professional and 
personal development. The project seminars, site visits, and emphasis 
on alternative assessment were considered especially valuable. Only a 
few minor criticisms of the AEDP project were expressed, and it was 
deemed worth repeating and expanding. (Appended is information about 
the study sample.) (MN) 
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I BACKGROUND 



1 About this report 

This is the report of an evaluative study, conducted over the academic year 
1991-92, of the Adult Educators Development Project at Lehman College of 
the City University of New York. The Adult Educators Development Project 
is a program of the Institute for Literacy Studies, Lehman College. The 
AEDP received its primary funding from the Fund for Improvement of 
Postsecondary Education (FIPSE) and had additional support from the Aaron 
Diamond Foundation, the National Center for Adult Literacy, and from local 
agencies in the New York City Adult Literacy Initiative including the 
Community Development Agency, the New York City Board of Education, the 
City University of New York, and the New York Public Library. 

The scale of available funding dictated a relatively *light' study. Data was 
collected by interview (with team and participants), some limited observation 
(of seminars), and study of documents. Observation of the participants' 
teaching and interviews with their students were not possible. Three visits, 
each extending over two or three days, were made to the project. 

Documents studied covered a wide range: project documents produced for 
team, participant, and external readerships; existing interview transcripts 
generated by an earlier study involving some of the participants; participants' 
letters and forms of application; and written work by participants (journals 
and research projects). 

The team was formally interviewed once. In addition I had regular 
conversation with them, face-to-face and by telephone, as well as 
communications by mail and fax. Two visit reports were submitted to the 
team for comment in the course of the year. Feedback from these helped to 
ensure that the questions I addressed included ones the team considered 
significant as well as those arising out of my own evaluation perspective. 

A sample of participants was constructed (for details see Appendix). These 
were interviewed as follows during the 1991-92 academic year: 

1989 intake 3 once, in February 

1990 intake 4 once, in February 

1991 intake 4 twice, in November and April 

The intention behind the last visit was to note changes over the course of 
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participation in the first-year AEDP program. In the report I have generally 
tried to preserve the anonymity of the informants. 

Parts of three seminars were observed at about the times of the interviews: a 
senmiar in the follow-up program of the 1990 intake was observed in 
February, and seminars in the current first-year program in November and 
April. 

Funding limits precluded the transcription of interview recordings. Instead, 
written notes were produced, at the time and from the recordings, which 
conveyed the sense and, over many stretches, the exact wording of the 
testimony. Where necessary the tapes were consulted for clarification and to 
supply gaps in the notes. Although it may be taken that the latter are a 
relatively faithful record, what informants told me in interviews will here 
generally be rqwrted as indirect speech, to avoid giving a misleading 
impression of verbatim quotation. This report was written from some hundred 
pages of such notes and a large collection of documents. (Citations from my 
notes are indicated with quotation marks, indentation, and references to the 
source.) 

In this report I seek to give a sense of the work and circumstances of the 
teachers who attended AEDP, and to convey something of the experience of 
the project as well as evaluating its effectiveness. Directly evaluative 
comment does not appear until well into Part 11, where the most important 
statements are printed in bold. These statements are gathered together, along 
with some recommendations, in section 8. 



In Lehman College and at the participants^ sites I was invariably 
received with warmth and hospitality. Teachers and project team 
members gave their time generously and did their best to help me, 
often going out of their way to find documents and sometimes 
answering my additional queries by mail. It was a rewarding and 
memorable experience visiting such a range of engaging and 
talented individuals working with dedication and often great 
creativity in the most diverse conditions across three of the City's 
boroughs. I thank them all for their help. 



2 The Adult Literacy Education scene in New York City 
2.1 The provision 

Like all large cities in the industrialised world New York has many adults who 
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lack the education necessary to obtain and keep jobs which demand more than 
elementary manual or interpersonal skills, or to gain access to and and benefit 
from college education. The problem in New York is more acute than in 
many cities because of the large number of immigrants who lack both 
education and knowledge of English. But even without this group the situation 
would be difficult: a disturbing proportion of the established population has 
inadequate education for contemporary needs. The lack is experienced as a 
problem not only by the city but by people themselves, many of whom actively 
seek education for goals which are personal as well as job-related; to be able, 
for instance, to read the newspaper or help children with homework. 

To tackle this problem help exists in the form of programs with various 
names, amongst them Adidt Basic Education (ABE), adult literacy education, 
and education in English as a Second or Other Language (ESOL). ESOL may 
be taught in the same program as basic or literacy education, or separately. 
By contrast with other education provided for adults in colleges and 
imiversities, these programs tend to be characterised (though with exceptions) 
by low status, poor resources and piecemeal provision. In New York City 
there are a number of provider systems, the sq)arate efforts of which are 
coordinated by the New York Adult Literacy Initiative. These providers are 
the Board of Education, certain colleges of The City University of New York, 
three library systems, and community-based organizations. Although there is 
variation in the nature and purposes of the provision, the goal is generally to 
move students through officially defined levels and into the General 
Educational Development (GED) program. The GED certificate is the 
objective for many students and is believed to bring access to college education 
and jobs. In more general terms the adult education programs seek to improve 
literacy and mathematical skills, and sometimes also social and life skills and 
knowledge for surviving in the society; many educators also involve students 
in critical examination of social issues. Beyond the basic level some programs 
working towards the GED follow the core of the high school curriculum. (In 
one program visited on the evaluation, for instance, each evwiing session is 
divided into three sections: the first hour is reading, with content from science 
and social studies, the second is mathematics and the third is writing.) Other 
programs incorporate vocational education, e.g. for clerical occupations. 



2.2 The teachers 

Of especial relevance for this rqport is the absence of an institutionalised 
profession of adult educators. There is no generally recognised pre-service 
training or accreditation. Few teachers have securit-y of employment beyond 
the current year (some Board of Education teachers are an exception); many 
are paid by the hour; others are unpaid volunteers. The prior experience and 
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qualifications of the teachers, and their motives for taking up the work, are 
correspondingly varied. In some sectors the teachers are a shifting population, 
for the most part remaining in a job, or in adult literacy or ESOL, for only 
short periods; other sectors are more stable. 

Given these circumstances, it is not surprising that there are worries about the 
general quality of literacy/ESOL provision for adults. The same 
circumstances, however, make it difficult to attempt system-wide 
improvement. In so far as the problem is one of teacher expertise, there exist 
neither an organizational structure through which a training program could 
reach all the teachers, nor the finance to support teacher time spent out of the 
classroom in in-service training. Where voluntary training sessions are offered 
they are often not attended by hourly paid teachers, who may lack the 
necessary motivation, or the time. One of the participants in AEDP, for 
instance, described how *she and her colleagues work thirty hours without 
preparation time and have to do other jobs in order to live. At meetings which 
she's attended, any ideas that involve more time are simply dismissed' 
(Interview). 



2«3 Assessment in adult literacy/ESOL 

Many programs set as objectives the attainment by their students of particular 
levels measured by state and city mandated tests: for beginning students the 
Test of Adult Basic Education (TABE) and for ESOL students the John and its 
replacement, the New York State Place Test. Transfer from the basic 
programs to GED programs may be determined by scores on the TABE or on 
GED practice tests. Within programs, students may be allocated to groups or 
^levels' on the basis of TABE. 

These tests, which in form are standardized measures designed to be 
administered to large numbers and to yield numerical scores, exert a powerful 
influence on teaching, and result in a greater resemblance between the 
programs than the institutional diversity would suggest. Accordingly, the 
assessment process is a point at which leverage can be exerted on teaching 
across the system, and attempts to change ass^sment may be motivated by a 
desire to improve teaching. There is a widespread view that the TABE and 
GED procedures can have anti-educational effects on programs, making it 
harder for students to acquire skills and knowledge of which they stand most 
in need. Reform of assessment, however, is sought not simply for its 
washback effect on teaching. There is concern also, felt by experts in the 
Institute for Literacy Studies amongst others, that the procedures are deficient 
as assessment and generate information which can be inadequate and even 
misleading. 
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3 The Adutt Educators Development Project: intentions and design 

The project on which this study reports arose out of an immediate concern 
with these assessment processes and a more general concern with the quality of 
teaching in adult literacy education and ESOL. The Adult Educators 
Development Project is described in the official proposal documents as 

... a collaborative staff development initiative for selected teams of 
adult lit^cy teachors working in four diverse settings in New York 
City: CUNY colleges, community organizations, libraries, and 
schools. Through activities including a staff development seminar, 
program-based student observations and interviews, and the 
development and field-testing of curricula and classroom-based 
assessment procedures, the AEDP seeks to enhance the professional 
development of literacy teachers while redefining and reforming critical 
relationships between curriculum and assessment in adult literacy 
education. 



The current project team are Marcie Wolfe, Director (on childcare leave for 
the period of the evaluation); Karen Griswold, Associate Director; Lena 
Townsend and Deborah Shelton, 1989-90 participants who joined the team 
during the second year. All have extensive experience in adult education. 
The Director has made explicit the project's theoretical position on literacy and 
assessment in a co-authored paper published by ERIC (Susan L. Lytic and 
Marcie Wolfe, *Adult Literacy Education: Program Evaluation and Learner 
Assessment'). The project's key ideas and positions will be amply illustrated 
in the material that follows and will not be spelled out at this point (see, for 
instance, 6.1, 7). 

History. AEDP's three-year program began in September 1989 and ends in 
August 1992. As was the intention, recruitment in each year drew from the 
four principal providers of literacy and ESOL education for adults: CUNY 
colleges, commimity-based organizations under the administration of the 
Community Development Agency, public libraries, and the Board of 
Education. (The community organizations represented were Bronx Educational 
Services, DC 37, Church Avenue Merchants Block Association, Henry Street 
SetUement House, Literacy Volunteers of New York City, the National Puerto 
Rican Forum, and the Young Men's/Young Women's Hebrew Association.) 
The 1989 intake comprised 19 participants, the 1990 20 and tiie 1991 21. 
Because lead-time before the start of the first seminar series was short, the 
team were compelled to select the initial intake without the careful review 
procedure which had been planned. As a result problems arose with this 
group (see below, 5.2) which were able to be avoided in the subsequent years, 
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when all applicants were interviewed. Administrators (the term lacks clear 
definition in adult literacy education) were among those admitted as 
participants, a decision taken by the team with some hesitation due to worries 
that their presence would inhibit teachers from speaking freely (see below, 
6.1). 

In order that participants would be able to attend the seminars on Fridays 
during working hours (as well as some in their own time at weekends), 
organizations agreed to fund their release. Participants were offered payment 
tor their curriculum development work, or tuition fee waivers to the same 
value for Lehman CoU^e graduate credit. I was unable to assess how 
important these rewards were as incentives to participate or persist in AEDP. 

The expectation was that the first two intakes would each continue their 
involvement for a second year. The numbers who did so (though not the 
experience they received) were a disappointment to the team. Of the 1989 
intake, 11 agreed to continue for a second year; of the 1990 intake five 
continued. A reason why many participants did not return for the second year 
was the difficulty of getting release time from their programs. One participant 
who did continue from the original intake said that even she and her colleague, 
who came from an organization strongly supportive of AEDP, had had to 
*sneak out' unofficially to attend the follow-up year's seminars. Some other 
participants, she thought, simply lacked the commitment to contiinue into the 
second year. 



4 The participants and their work 
4.1 Jobs, careers and work contexts 

The participants I studied worked at the following jobs. Two worked for 
Board of Education Basic Education programs. One taught a Board of 
Education clerical, pre-vocational education program. One taught for sixteen 
hours a week at the Adult Learning Center of a CUNY college. Two worked 
as ABE (Adult Basic Education) instructors for a local community 
organization. One of these had a second job as an ESOL teacher for the 
program in which another participant was an administrator; in ABE she 
prepared students for the GED in a program aiming to make people more 
employable. One started the year with a community organization teaching 
ESOL and basic education but moved in January to the CUNY college Adult 
Learning Center at which the participant mentioned above already worked. 

The jobs of the participants who were administrators were as follows: one 
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worked for a community organization mainly as coordinator of two sites (she 
also did half a day's teaching at another site and worked one day a week as a 
teacher for a different organization). A second coordinated a community 
college program preparing people for employment (see below, 4.5). Finally, 
two wCTe site advisers for the Centers for Reading and Writing, The New 
York Public Library. 

The diversity of career background of the eleven teachers was typical of adult 
educators generally. Some had qualifications, with or without experience, in 
elementary or high school education; several had worked in government or 
commerce; one had been a musician, one a community worker; one was a 
writer. None had come straight to adult education from coUege; all had 
entered the field as a result of chance opportunities, of experiencing and liking 
some other sort of work with adults, of dissatisfaction with their existing job 
and consciously looking for an alternative, or from part-time volunteer work as 
literacy tutors. 

The physical and institutional settings in which the eleven teachers worked 
varied a great deal, though the situations and backgrounds of their students had 
much in common. Five brief descriptions will serve to indicate the range and 
the commonalities. 

Part of one person's job in the Adult Learning Center of a CUNY college was 
the coordination of the Student Literacy Corps Project, a US Dqpartment of 
Education initiative which involved training undergraduate students in the 
college to work as tutors to a group of adult students who were producing a 
community newspaper. For her other three days this teacher taught a mixed 
Level 2-4 class comprising mainly Latina women whose youngest children had 
now started school and who came for improvement in reading and writing. 
Some, mainly the men, had the GED as their goal. Although they had job 
aspirations, the teacher saw their main ambition as a more general desire to 
cope better with everyday demands, know more and understand the world. 

The teacher who had recently moved to the Center had been teaching ESOL in 
a quite different setting. Her students had been mainly Latinos (and a family 
of three Senegalese) and a more varied basic education group, some with 
Spanish as their first language, some who grew up in the United States, and a 
couple from Jamaica, West Indies. The suggested duration of the program for 
individual students is three years; GED is a distant prospect for most. The 
site where I first talked to this teacher, like the sites where several participants 
worked, was in a poor area where visitors like myself were advised to exercise 
care for their safety. 
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A third participant taught her clerical, pre-vocational course iii a shabby, 
though once elegant, typing room in an old school building in a nm^lown 
area. The students, mainly mature women with low scores on the TABE test, 
wished to gain skills which would give them access to better jobs. This 
teacher was generally successful in getting the students within a few months to 
the point where they could join the GED class. 

A fourth teacher taught in a community based education project in what had 
been a stylish old parsonage in another poor area. The objective of the new 
program on which she taught was to get student- o a stage where they could 
handle entry-level jobs. The students, some twenty of them, all women, had 
been required to attend and given travel money to the program by the agency 
which administers their welfare payments, the Human Resource AdiHinistration 
of the Office of Employment Services. The situation of these students was 
typical of many in adult literacy and ESOL programs. All were mothers, 
mostly single parents, in their late twenties to early forties, some with grown 
children. This was their first experience of education since dropping out of 
school, where they had acquired a sense of themselves as inadequate and 
unable to learn. In the teacher's words, they had arrived flustered, confused, 
anti-system, and with some hostility, saying *Here I am again, back in school'. 

The 36 hours a week which a fifth teacher worked for the Board of Education 
were divided between literacy in the mornings and, in the afternoon, GED, 
which he preferred because of its content base and the social issues it allowed 
him to bring in. His site was, in his account, isolated, *in a project at the end 
of a bus line where middle class folks don't come'. 



4*2 Teaching adult Uteracy/ESOLS 

From what the teachers told me and wrote for their AEDP assignments I was 
able to learn something of how they went about their job of teaching adult 
literacy/ESOL. There was wide diversity, both in reported practice and in 
articulated rationale. For example, the mechanics of writing were generally 
taken seriously, but attention to these was associated in some cases with tightly 
controlled and bounded work (e.g. writing single sentences, reading in 
sequence through all the books in a published scheme) and in others with 
activities in which students had choice and could engage with themes that were 
important to them (e.g. reading chosen books on Black history or writing on 
issues such as bringing up children). The two tendracies seemed to reflect 
attitudes and beliefs about the students, emphasising on the one hand the great 
difficulties they experienced and on the other the inner resources they brought 
with them. (It was clearly an intention of AEDP to swing participants towards 
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the latter position, or to confirm the convictions of those who already espoused 
it.) 

Characteristics of ^proaches consistent with AEDP thinking (though not 
necessarily derived from AEDP) were: 

Seeking out materials with which to replace published reading schemes^ 
to reflect participants' concerns and life-themes and include a wide 
range of genres. A teacher, for instance, who several times mentioned 
this task as the one that most preoccupied her contrasted the materials 
she needed (e.g. on Black issues) with GED reading matter which she 
saw as oriented towards middle-class values. Unusually in ABE/ESOL 
programs, her students read poetry, in which they *at first found littie 
sense but now actively sought meaning' (Interview). 

Working to overcome students' fear and distrust of education. This 
might involve making a supportive community out of the group: the 
class of the teacher just referred to had after five months become an 
interactive network that provided affective satisfactions and a focus of 
social activity such as shopping together. More centrally it involved 
giving students confidence that they could learn and experience of the 
enjoyment of learning: in that class and in some others the initial 
bre^cthrough had come in mathematics; in one it had come with the 
writing of an autobiographical piece. 

Some teachers believed that a source of their effectiveness was their 
communication to the students of a sense of shared experience. One 
teacher 

* considers he could have been a writer but the model of his 
father, a journalist, militated against his finding a voice of his 
own. Now, although he feels more confident about his own 
writing, he says his desire to find a voice still coincides with 
that of his students; he is no different from his students. He 
discusses this with them all the time: they see him as an author. 
They need warmth and support. The group is like a working 
family' (Interview). 

Combining attention to form (especially spelling) with the making and 
communication of meaning. One teacher, for example, moved in the 
account she offered of her work over the last couple of weeks from 
^having the students write words that they wanted to leam, then asking 
tiiem to compare words witii similar patterns like "flame" and 
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"blame"', to ^listening to their conversations outside the classroom for 
themes that could come up and be dealt with and written about in class, 
things that are important to them': for example, childhood, and what 
makes people tense (Interview). In another class, *The topics are ones 
of their choice and they really do talk and write about topics of 
concern, for example, the homelessness, crime in the city, raising 
children' (Interview). The topics students chose in these programs 
were not only current issues: one class wished to study Ancient Egypt, 
so the teacher *has been learning all about the topic over the vacation 
and has found useful mileage for her literacy work in the stuff about 
hieroglyphics' (Interview). 

Providing plennfid experience of continuous reading and writing. *The 
method is basically to get them to do it and to provide strategies and 
techniques. Getting them to sit there and seriously read and do writing 
is the main part. Help can be anything from providing time and 
opportunity to providing a word' (Interview). 

The use of reading and writing to support each other. * Collaborative 
writing is used as a reading text for the group. If an individual seems to 
have liked the text, they write the version which they would like. It 
helps that everybody has a piece of text by them when they're writing: 
it alleviates the fear' (Interview). 

Recognising that aduh learners were different from elementary and 
school students, even though what was being learned might sometimes 
be similar. Such a recognition entailed more democratic ways of 
proceeding: thus one teacher structured times for collective decisions 
about the ways the class was run. It also entailed a different attitude 
towards attendance and punctuality. As she explained: 

'with these adults you can't say, "You have to be here," as you 
might to children. Nor can you say "This is like a job and we 
expect punctuality and attendance". This sort of context has to 
be structured in quite different terms from the so (ool. It's more 
fluid and less rigid. The people we're working with have cares 
and responsibilities and cannot be accused of non-commitment 
because they don't attend regularly' (Interview). 

While the most experienced teachers had no shortage of strategies and 
confidence in tackling the formidable problems they faced, the inexperienced 
teacher operating in isolation with a minimum of support could feel 
overwhelmed by the difficulty of the task. In such a situation one teacher, 
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perhaps needing a strat^y which was definite and concrete and would (as it 
seemed) allow progress to be clearly seen, relied heavily on a published 
reading scheme and teaching of formal features (word meanings and spellings, 
word endings, singular and plural, possession, writing personal and business 
letters, and grammar, including parts of speech (which she considered hard to 
teach to adults: *They know it but then they use the wrong part anyway'). 
This seemed an example of the teaching style cited as a problem in the original 
AEDP proposal: *In this model, reading and writing are thought of as being 
learned from practice with their component "skills". In reading, this subskill 
approach may include practice with recognizing main ideas, inferring 
meanings, or memorizing vocabulary; in writing, it may include practice with 
words and types of sentences' {Proposal^ p. 2). 

This teacher's percq)tion of her students was dominated by their apparent 
inability to leam: 

*They have real problems and she stressed that it was very difficult 
teaching them. They don't remember a word from one time to the next 
and she wonders whether they will always be iti her class and never 
make progress out. It's very hard to get them to make progress. 
There's a great deal of absence, some of it for personal reasons but 
some of it just lack of commitment. This makes it very difficult to 
work with a whole class as a class' (Interview). 

The latter observation contrasted strikingly with another teacher's testimony 
that regular attendance in class had risen from 2-4 to 15-18, as a result of 
effective teaching. 

Even for experienced and successful practitioners, following the educational 
principles indicated above was not straightforward. A pedagogical dilemma 
which came up in several interviews was how far respect for students' 
perceptions and wishes should be extended to their views of appropriate 
curriculum and pedagogy. While teachers might seek to offer a wide range of 
learning experiences, students often objected that some were not relevant to 
preparation for the GED. One teacher's approach was to talk them out of this 
attitude and convince them that everything they did was a preparation, more or 
less direct. Another experienced a real tension in this area, while 
acknowledging that the problem was probably less acute in her context, the 
public library, than in Board of Education settings since in a library people did 
not expect a structured classroom. She tackled the students' educational ideas 
directly by enlisting experienced si : lents to talk to them about learning 
techniques. 
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Others, however, felt it was necessary - tactically but also perhaps on 
principle — to give some acknowledgement to the rights of students to 
determine what was good for them. Thus, although one teacher 'does not 
share her students' enthusiasm for dictation, she feels that because they think 
it's important and get a great sense of accomplishment when they've mastered 
it, she has to provide it' (Interview). Another straightforwardly 'gives the 
students what they want as well as pursuing his own agendas. Tliey want 
spelling so he does it Mondays and Fridays, with a test each Friday' 
(Interview). 

Members of the group had discussed the issue among themselves. One 
participant described how she 

'had been disturbed that students in her basic education class were 
rebelling and saying they did not want to take the local planned hospital 
waste incinerator as a crucial issue. They hadn't come to school for 
this; it was boring, their opinions wouldn't make a difference. She had 
asked another participant how to convince the students, and had 
received the response that these were adults who knew what their 
priorities were and that we should accq)t their agendas' (Interview). 

A frustration which participants experienced with the standard tests was the 
inability of these to register the types of progress which were most important 
in day-to-day work with students. An example of achievement which was 
highly significant to the teacher but which would pass unrecognised by the 
tests is contained in the following testimony: 

'Some students are terrified of writing. One student would not let [the 
teacher] see the drafts for three months. When that battle was won it 
was a liberation. [The teacher] had to persuade her hard to invent 
spellings. A student who three years ago was writing two sentences 
now writes pages of well-<:omposed text, small essays, about her life. 
[The teacher] doesn't know exactly how she got there. It's to do with 
an increase in confidence and the belief in having something to say and 
also support from the class. She's also reading better. Punctuation and 
spelling are still not right and syntax is not standard English but the 
writing is clear. When she learned in this cycle to understand the 
paragraph, that was an illumination. She was then able to organize her 
thoughts in a more complex way' (Interview). 

Such stories help explain the strength of the desire for forms of assessment 
capable of recognising teachers' and students' significant successes. 
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4*3 Control over teaching 

None of the teachers complained that restrictive program policies or 
supervision prevented them from teaching the way they wanted to. In some 
cases it appeared that management did not have the resources to oversee the 
teachers in any case. Supervision could be intermittent (and not necessarily 
helpful): *The coordinator visits from time to time and, when he observes, 
requires that [the teacher] teach a whole class lesson. But mostly nobody tells 
you what to do' (Interview). Some participants, however, felt, positively, that 
they were encouraged to take initiatives. Typical testimony was: 

*She finds there's little restriction from her superiors. She has a lot of 
freedom. Teachers are encouraged to devise their own methods' 
(Interview). 

*She has considerable freedom to write her own program. There are 
few constraints on her if she wants to change or iimovate' (Interview). 

The team described to me two types of institutional constraint which I did not 
encounter in my interviews. One was the strength of existing institutionalized 
routines: a group who had attended AEDP from the same program seemed 
simply to have confirmed each other in practices they had already established. 
The other was exemplified by 

a success which caused problems: a participant in the first group 
had come to question the practices in her own institution and indeed 
had involved the students in this questioning. As a result she had come 
up against the program's management and finally had to leave. She no 
longer works in Adult Basic Education' (Interview, project team). 



4.4 Assessment in the teachers' work 

Assessment as a requirement. The standardized TABE and/or GED tests 
were the official determinants of success in a number of programs. ABE 
students above the third grade level take the TABE at intervals, and ESOL 
students the JOHN test, now being replaced by the New York State Place 
Test. The tests were also widely used for the assessment and placement of 
new intakes. Students also took an entrance test to move from ABE or ESOL 
to GED programs (the test might be a practice version of GED). The only 
program, of those my sample represented, in which these tests were not 
significant goals was the New York Public Library tutorial program (as 
opposed to the classes which the library also ran); the TABE test which that 
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organization had formerly chosen to use (though not required to) had been 
dropped. 

Reservations which participants expressed about the tests were as follows: 

The GED: said to show middle class bias, to be based on reading 
alone, and to fail to encourage critical thinking. Used as a summative 
measure it could give misleading indications of what a student could 
do, as in one program in which students made their own decision about 
when to enter themselves for the GED: here even ABE students 
assessed by TABE at Level 1 had passed (Interview). 

The TABE: 'TABE does tell you something (Level 2 really is different 
from Level 9). The trouble with it is that those who are good at it 
crow' while the rest are demoralised (Interview). 

Nationally standardized tests in general, used for placements, *do not 
satisfactorily indicate where students should be placed. The test 
situation makes the students tense. [Informant's] experience is that 
people who score quite highly may still have major difficulties which 
the tests fail to pick up. And they say nothing about writing' 
(Imerview). 



Teachers^ own assessment practices. Whether or not their students were 
submitted at key stages to the standardized statewide tests, teachers often 
employed forms of assessment of their program's or their own devising. At 
the sites which she coordinated, although her agency required TABE as the 
instrument of placement and assessment, one administrator assigned students to 
their writing class on the basis of a writing sample which she assessed in a 
holistic way in consultation with the tutor. In this program there was a 
general testing, open to any member of the public, one week before the end of 
each term and one week before the start of the new term. 

A problem faced by AEDP was the mistaken belief of some participants that 
certain assessment practices they maintained were consistent with the principles 
the project was recommending. One participant, for instance, had been an 
active designer of assessment tools within a particular vocational area; an 
example was ^simple' lists of subskills which she claimed a new teacher could 
easily foUow but which, in the AEDP perspective, might be seen as failing to 
take account of the integratedness of the operations involved. Again, a 
teacher's routine daily notes which she referred to in the interview as an 
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example of on-going assessment in the spirit of AEDP, turned out to be 
numerical grades; the criteria were written up in booklets for her colleagues 
(who, however, had rejected their application as too elaborate). Self- 
evaluation by students is a procedure recommended by AEDP, but the printed 
form used for the purpose in one classroom, while appropriately inviting 
students to name their most successful areas, what they enjoyed and didn't 
enjoy, the benefits they were getting, and additional subjects they would like to 
see, left only small spaces for writing, despite this teacher's acknowledgement, 
in interview, of the value of extended writt^ comment. 



4.5 Administrators' work 

Although other participants had some administrative responsibilities alongside 
their teaching, four of those I interviewed were primarily administrators. 
They had very different problems. For the first of these the preoccupying fact 
of life was large numbers of students and restricted hours of teacher time with 
which to meet the students' needs. 

*Her supervisors see eye-to-^ye with her and invariably agree to 
anything she puts forward as long as she makes a good case. They back 
her with resources; she has all the books that she wants and they're not 
checking up on her all the time. The problem is rather the shortage of 
time with the classes' (Interview). 

Although as coordinator she is not formally responsible for the instruction 
offered in the program, because she is interested and has considerable teaching 
experience she chooses to meet frequently with the teachers to talk about their 
classes. Her scope for improving the program is, however, severely limited: 

'None of the four instructors are full-time — she's the only full-time 
person. No one arrives or is employed before 4:30 p.m and instruction 
starts at 5:30 so scope is very limited and there is nothing she can do to 
pay people for extra hours' (Interview). 

As coordinator of an employment-oriented program in a community coUege a 
second participant supervises twenty-five teachers. This program seeks to 
teach English as a second language with a vocational emphasis. The 
administrator finds it a frustrating and difficult program. Attendance for the 
students is mandatory. After eight weeks of immersion, the mainly female 
Hispanic students spend five months out at a work site where they often have 
little opportunity to develop their English* The administrator's professional 
development work with the teachers, however, is rewarding and effective. 
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The environments in which the two New York Public Library site advisers 
worked offered a striking contrast with the run-down buildings and depressing 
circumstances with which other participants had to cope. The spaces used by 
staff and students were relatively well maintained, and well resourced with 
equipment, furniture, books and materials, and clerical support. One 
administrator was responsible for recruiting, teaching, and assessing students 
and helping them to assess themselves, and for recruiting, training and 
supporting volunteer teachers. She was also in charge of computer instruction 
and advice. She was thus not purely an admirdstrator but had a a great deal of 
contact with the students, not least because everybody at her site worked in 
one big room. Her colleague in the Public Library coordinated staff 
development and general management in eight centres in three boroughs and 
was the specialist in learning technologies (computers and library materials). 



4«6 Professional development 

Between them the participants had taken a large number of potentially relevant 
courses at different points in their careers. Mtoily these had been education 
courses, often in reading. Although frequently mentioned as useful, however, 
few courses had addressed issues specific to adults. Two universities were 
mentioned as offering Master's degrees in AE, but no participants had taken 
them: one said he could not afford the fees. None of the participants had had 
specific training in adult education before entering the field, beyond a brief 
orientation. (Preparation for one participant had been a single-day orientation 
course which explained the difference between adult and child education.) The 
sources most often cited for participants' expertise were more experienced 
colleagues and supervisors. 

Some of the organizations supported staff development, typically in the form 
of practitioners' meetings within or across sites, as often as once a week. 
Some of the administrators of the project were responsible for organizing these 
(in the community college and the library). The teachers found them useful. 
In the view of the project team it was paradoxically the professional teachers 
who sometimes got the least sophisticated provision; volunteers in some 
organizations were better looked after. 

^Provision was often on the single workshop model without the 
opportunity for extended reflection on practice, although there were 
exceptions to this. Most teachers in adult literacy might go to a few 
workshops each year. But the opportunity to work over a whole year 
is very unusual. Organizations don't on the whole make such 
investments for teachers who are generally not expected to stay in the 
field for very long' (Interview, project team). 
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In addition some participants had attended open workshops run by various 
organizations, and the annual New York ABE Conference. 
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4 J Involvement in AEDP 

Applying to join AEDP had required the support of employers since 
participation involved release time from work. In some cases the 
organizations, having received the AEDP notice, had proposed to the teachers 
that they apply; in others the teachers had taken the initiative. In some 
organizations more individuals wished to participate than the employer was 
prepared to release, Participarits' motivations for joining were varied: a 
concern with assessment was usually a strong factor; another was knowledge 
of the Institute's work and an expectation that the program would be 
rewarding. 

^Although testing had just stopped in their program [the interviewee] 
was still concerned with genml questions of assessment, and this was 
also the next issue on her program's agenda. She was concerned to put 
assessment in the hands of the students and to make it a negotiated 
business. The teachers felt they knew that their students were learning, 
but only in general terms, e.g. that they were reading more and were 
more critical. She wanted to be able to generate more concrete 
information. A particular question was "Are the students still looking 
at learning in a traditional way or are they internalizing our 
approaches?" She hoped that these issues would be looked at more 
rigorously in the context of a serious study group' (Interview). 

The interest of her colleague, who had been equally keen, had been in ^getting 
people to understand what they'd learned and to look ahead and be able to 
articulate their learning process' (Interview), Another administration (the 
Board of Education) had also 'declared that alternative assessment was an 
urgent issue and were themselves starting a workshop [The interviewee] 
hoped for guidance on how to evaluate students' writing holistically' 
(Interview). Another participant, one of two sent by the same agency, *was 
already thinldng about assessment because he had been to a one-day session at 
Lehman' (Interview). 

Two teachers in a community organization had seen the brochure and asked 
for permission to attend, motivated by a general interest in the topic. *[The 
interviewee] applied to gain a better idea of how to do assessment in a way 
that was acceptable both to her and to the students and out of a dislike for the 
standard assessment procedures. She was aware that she was unfamiliar with 
alternative forms of assessment .... ' (Interview). A participant from a 
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different organization had applied for similar reasons, 'at a point when she was 
confused and worried about a new burdensome test which had been imposed 
on her program. She was hoping that the course would give her information 
about other options' (Jmerview). Yet another had also applied on her own 
behalf *out of hunger for knowledge. She was not specifically interested in 
assessment but had been impressed with the Lehman College team and liked 
the Adult Learning Center' (Interview). Finally, a participant now in her 
follow-up year, had implied for a quite differrat reason: •her site team had 
been broken up and scattered so that she was left teaching alone with one other 
person. A strong motivation for joining was to meet with colleagues' 
(Interview). 



n THE WORK OF AEDP 
5 AEDP^s activities 
5.1 Theory and practice 

AEDP existed to change practice. Thus the role in the program of ideas, 
theoretical and practical, was to inform what teachers did in their classrooms. 
Even, however, had the focus been centrally on theory, pedagogically it made 
for effective learning if participants thought about and attempted practical 
applications, as well as reflecting on the ideas in talk and writing. Central to 
teachers' participation in AEDP, therefore, were investigation and experiment 
at their own sites. 

AEDP set out to affect the practice of, at any time, some two dozen 
participants who worked at widely scattered sites. To achieve this the project 
had at the most three staff, none of whom worked more than half-time on the 
project and who clearly could not work alongside the participants in their 
classrooms for more than an occasional few hours. The various project 
activities may be seen as a response to the challenge of changing or developing 
practice at a distance and by largely indirect means. It is a challenge faced, of 
course, by most forms of teacher training. AEDP's solutions, however, were 
innovative, certainly when seen against teacher education in general, though 
some had previously been used in projects within the Institute for Literacy 
Studies. 

The traditional teacher training model is to impart ideas in an academic setting 
and trust that they will subsequentiy inform practice. AEDP, in con rast, 
employed various strategies specifically to address the transfer of ideas from 
academic to program settings. Perhaps the most basic of these was to bring 
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the teachers' own practice into the seminar as content for examination: the 
efficacy of the team's theoretical framework was then demonstrated less by 
exposition than by the light it threw on the teachers' experience and students' 
work. 

The basic elements of the experience provided by AEDP will now be 
described, with some evaluative comment gathered from participants. 



5.2 The sminar 

For this section I draw both on participants' accounts and on my observation 
of parts of two first-year seminars of the 1991 intake group and one follow-up 
seminar of the 1990 intake group. 

Seminars at the Institute for Literacy Studies in Lehman College were the 
occasions when the group met and were thus the central element of both first- 
year and follow-up activities with each group. They were generally held on 
weekdays, twice a month with the fiirst year group, with three additional 
Saturdays, and monthly, with some Saturdays, in the follow-up year. The 
seminars fulfilled a variety of functions which the organizational forms 
reflected. The latter included whole grour meetings (presentations, 
discussions, exercises); small group meetings (regular groups of constant 
membership, focused on sharing journal writing, and special groups 
constructed around research interests once individual projects had got under 
way); and one-to-one consultations between a team member and a participant. 

The two fiirst-year seminars which I observed in part were structured as 
follows: 

'This session was mainly about interviewing as a technique of 
alternative assessment. The plan was fiurst to discuss the business of 
interviewing in general (this was going on when I arrived), secondly to 
generate a list of questions for interviewing each other, thirdly to 
categorize those into groups, fourthly to do interviews and fifthly to 
reflect on the experience and consider implications for interxdewing the 
students. Time ran out before the last could be accomplished. Near 
the end of the session, a homework was to conduct an interview 
with a student and write it up. Participants were to bring five copies of 
the transcript to the next session. An article was also distributed to be 
read for the next session' (Seminar observation, November). 

*Karen had explained holistic scoring of writing [in the context of a 
discussion of direct and indirect measures of writing]. There was 
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discussion of the grading by ^)ecified characteristics (rather than by a 
'sense of a grade or lever). Discussion of the research projects 
followed. The seminar was to continue the next day (Saturday), with 
discussion of Linda Brodkey's article *The Literacy Letters', and with 
presentations by two participants about the data they were collecting' 
(Seminar observation, April). 

The functions which the seminar served could be summarised as foUows: 
bringing together teachers from different organizational and institutional 
contexts; demonstration of techniques of teaching, assessment or research; 
consultation between team members and individuals about journals or research 
projects; reviewing work done individually since last meeting, and plans for 
the next stage; and discussion of articles read. Thus a great deal went on in 
the seminars — more, in the project's view, than could be taken in and 
evaluated at the time. Consistently with their general emphasis on the power 
of written records as a means of later learning, the team accordingly required 
participants to take full notes. These were reflected on subsequently in 
writing, normally on a side of the page which had been left blank for the 
purpose. 

Learning from others. The format of the seminar was designed to maximise 
interaction. As weU as the use of a variety of meeting formats, as mentioned 
above, lunch was planned as a pleasant social occasion: the group stayed 
together in the seminar room and shared food which they had brought. 
Participants frequently stated in interview how they valued the chance to meet 
practitioners from other sites. Outside AEDP there were few other 
opportunities for interchange with teachers doing related work in other 
organizations. There w^e three principal benefits. First, the teachers 
acquired a sense of the scope of the work of their own profession and of the 
range of contexts in which it occurred. Since Adult Education is undeveloped 
as a profession, raising standards is seen by AEDP as in part a matter of 
building a professional community with a sense of identity, a body of 
professional knowledge and a research tradition. Bringing teachers together 
from different organizations and types of institutions was seen by both team 
and participants as an important step. Second, participants learned about 
teaching and assessment ideas employed by colleagues at different sites, and, 
third, they helped each other in tackling the new ideas and techniques which 
AEDP presented to them. 

Learning was found by participants to result not only from exposure to new 
ideas but also from the need, which arose from the diversity of the group, to 
articulate and justify one's own taken-for-granted assumptions: 
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*A11 have similar experiences. But there is a variety of views, some of 
them more traditional, though the^ are ^.xpressed without conflict. 
This is healthy because when someone has a different viewpoint, one 
has to formulate one*s own ideas more sharply, and back them up. 
Some find the sessions more exciting and meaningful than others 
because of different starting points and openness to ideas. The richest 
aspect of the seminars has been that people have stimulated each other. 
All get ideas from others' discoveries' (Imerview). 

The benefits of collaborative learning could be experienced in both large and 
small groups. An instance of learning in the full group was described by one 
teacher: 

^[One of the participants] read a piece from a student whose mother 
died when she was young - very powerful and simple. Everyone in 
the room said something about it and it was amazing what was learned 
about the text. It illustrated the fruitfulness of looking in the first place 
not for error but strength' (Interview). 

The same episode was referred to also, in similarly positive terms, by another 
teacher, one who was generally not uncritical of the project- She 
acknowledged that 

'in examining pieces of writing she tends to emphasise their 
shortcomings and doesn't sufficiently stress their positive aspects. The 
fiirst time the latter approach was practised with Debbie in the seminar 
it came out so differently; she had to sit and rethink' (Interview). 

While observing a group discussing their research projects I noted 

*The writing group I'm with accepts a suggestion from one of them that 
they help each other with coding their data, and a proposal from 
another that they take one data sample and code it together. They 
proceed to do this' (Seminar observation). 

The second teacher leads the coding exercise. As a specialist in business 
education who felt generally isolated from the educational mainstream, she was 
gratified that her expertise was found useful by others. She was able to share 
what she had learnt about analysing and coding data, first in this small group 
and on the following day in the full seminar. 

Imparting techniques. Considerable seminar time was devoted to the 
teaching of techniques, for teaching, assessment and research. Extensive 
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preparation was given, for instance, for the research projects: at the time of 
tiie April interviews the team had, according to a participant, showed 
participants ways of working with data, finding themes and issues, and 
developing questions. Earlier (as reported above) I had seen an extended 
introduction of techniques and issues relating to interviewing- 

Review of assigned activities. Between seminar meetings participants were 
expected to engage in various activities, some on a regular basis (reading, 
writing a journal), others for a specific purpose to do with a current topic. At 
a seminar, for instance, I noted: 

*Near the end of the session, a homework was set to conduct an 
interview with a student and write it up. Participants were to bring five 
copies to the next session. There was also an article to be read' 
(Seminar observation). 

Time in the seminars was regularly given for reviewing work done at the sites 
since the last meeting. For example, in the follow-up seminar which I 
attended 

*two participants were asked to share some data they'd brought. [One 
teacher] read out her observation notes about a socially tense situation 
that had arisen between some members of her class. A useful 
discussion followed in which Lena pointed out that it was [the 
teacher's] engagement in this writing that had caused her to see the 
issue of whether a particular student was jealous. Without the writing 
this awareness would not have arisen' (Seminar observation). 

Discussion of articles read. In accordance with AEDP's general belief that 
learning was enhanced by deliberate engagement in reflective practices, 
assigned readings were gone over both in the participants' journals and (where 
time allowed) in class. On one of my interviewing visits on a Monday, three 
participants were still actively reassessing a discussion which had taken place 
in the previous Saturday's seminar of a paper by Linda Brodky. The group 
had been divided in their response to the article (the divisions centering around 
whether the racism which the author had set out to overcome in her class had 
in fact persisted in her own attitudes). 

The conduct of the seminars. The team sought in the seminars to follow the 
same procedural principles which they advocated that participants use with 
their students. These principles included a democratic receptiveness to 
learners' wishes and concerns; and the pedagogical principles that teaching be 
based on adequate knowledge of the learners' conceptions and that learners 
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have extensive opportunity to process new ideas in their own language, in 
writing and discussion. The seminar discourse was thus highly interactive, 
with only brief stretches of monologue by team members, (An occasion when 
Karen gave a talk about holistic assessment [Seminar of 10 April] was noted 
by one participant as unusual.) Maintaining this approach could involve 
tensions, as when participants had a view of teaching in which the authoritative 
delivery of e?q)ertise figured more prominently, or simply preferred to be told 
more, and when the prior knowledge and ^processing needs' of participants 
varied widely. This was particularly the case in the first year of the project. 

It was clear that the nature of the first group, recruited without the benefit of 
interviews and arriving in some cases with false expectations, had made the 
1989-90 seminar a difficult experience for the team. The participant's 
comment, quoted above, on the exchange of diverse views without conflict, 
would not have been made by the first AEDP participants, who experienced 
considerable conflict, ascribed variously in the interviews to Apolitical 
activists', to incompatible personalities and to underlying class and race 
tensions. In comparison with teachers in mainstream education, adult 
literacy/ESOL education does appear to contain a higher proportion of 
individuals who see themselves as marginal to the institutions, who have 
entered the field out of a passionate sense of justice and who have hitherto 
worked mainly in freelance and non-bureaucratised types of jobs where 
collegial discussion on the seminar model is not a routine experience. At all 
events, according to a participant who had continued into the second year, 

*There was a wide and wild range of participants. Some of them had 
far-out ideas about the instruction of adults, referring to them in terms 
like "those people". It was a bizarre group. After the initial meetings 
when four or five dropped out and the group reduced to about twelve it 
was a lot calmer but still got quite heated because there were very 
different perspectives. The second year was more enjoyable, with four 
or five people only' (Interview). 

The team's procedural assumption that the project was a collaborative 
exploration did not initially prove a viable basis for getting such a diverse and 
conflict-ridden group to function effectively. With benefit of hindsight, 
another teacher who participated for the full two years said it had been a 
strategic mistake by the team to operate so democratically, by discussion rather 
than by giving information: 

'In [the interviewee's] view, the team had been too kind. They had 
wanted it to be collaborative but the group didn't know enough to 
collaborate effectively. They had needed a lot of information first but 
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instead there was a lot of abstract discussion. It had been very up in 
the air. It had also provided a soap box for political activists. When 
these people dropped out, [this teacher] had stuck to it, though doubtful 
about its value. By mid-way, however, the community building 
process had worked and it had become good. They had started 
working on projects. There were concrete issues at stake. They 
learned more from each other and the facilitators also gave more 
information' (Interview). 

The teacher quoted previously had also observed that the first year course had 
been *a bit touchy-feely or amorphous. She had wanted to dive into the issues. 
She acknowledged, however, that this was not to say she would have been 
capable of doing so' (Interview). 

That participant's opinion that what was needed by the first group was more 
information is not the only possible view. It could be argued that teachers 
working in adult literacy/ESOL already had the necessary knowledge, or that it 
could be gained in their own classrooms by observation and reflection. AEDP 
was essentially about bringing different insiguis and perspectives to bear on 
knowledge and experience that were already shared. These could not be 
imparted by an authoritative lecturer but had to be constructed by the 
participants themselves through reflection and discussion, albeit with 
instigation from ideas and opinions that might be presented in more traditional 
pedagogical modes. 

The 1990 and 1991 sessions began with carefully selected groups which 
proved more harmonious. The individuals I spoke to had enjoyed their first 
year far more than had the original cohort, and found the seminar a 
stimulating and rewarding experience. In addition, the team avoided strategies 
which had been less successful the previous year. Dissatisfactions with the 
management of the seminar were minor, such as one teacher's view that the 
use of time could have been tighter. (Though she acknowledged it was hard 
for the leaders to be highly prescriptive with participants who were really their 
peers, she felt that exercises in the whole group sometimes took too long. 
*Time has a different value when (for a Saturday) one has to make complicated 
baby-sitting arrangements' - Interview.) 

In general the s^mmars were highly valued by participants and met several 
of their most ui^ent needs: for meeting colleagues from different sites, for 
sustained examination of their practice and for exposure to and communal 
reflection on ideas from sources outside the city (particularly through 
articles). Allowing teachers to meet on a regular basis for extended 
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discussion, organized and informal, was perhaps what participants valued 
the most about AEDP. 

The informality and coUegiality of the management of the seminars made 
them enjoyable and minimised for less experienced participants any 
anxieties about contributii^ to discussion in a lai^e group« Thou^ the 
approach was at times difficult to handle, it is hard to imagine how a 
more prescriptive or tightly scheduled approach could have achieved the 
same ends. 

There may be room, however, for some tactical concessions to 
conventional expectations in the eariy weeks with a group. The team were 
right in principle to resist pressure to collude with some participants' 
demands for a heavy loading of information. Conscious thought do^, 
however, need to be given to overcoming participants' early anxieties 
about whether their investment of time and energy will be worthwhile. 
Helpful strat^es might include demonstrating at the start that significant 
work existed in alternative assessment and that participants would become 
familiar with it, and, second, creating a sense of businesslike briskness in 
the proceedings. 



5.3 Research projects 

In both the first and the follow-up year participants were required to conduct a 
research project and write a paper. The research project was to be an 
investigation into an assessment practice they had experimentally introduced. 
Titles of reports which I read were: ^Writing Assessmrat in a Multi-Level 
Class*, *Oral History Project*, *A Study of the Attitudes and Beliefs of 
Literacy Students*, *Using Alternative Assessment Methods during Student 
Centered Instructional Projects*, 'Self-Assessment as a Dialogue between 
Student and Teach^*, and * Writing Process as Assessment/Evaluation*, 

The initiation of the projects was preceded by ample discussion in the seminars 
and supported by readings on qualitative research, sheets of guidelines, small 
group discussion, one-on-one consultations and a series of advisory and 
compulsory deadlines. 

Participants in the current first-year group discussed thdr research projects in 
small groups based on shared areas of interest, namely: writing assessment 
(portfolios and related approaches); reading (reading/writing connections, 
reading strategies); documentation for influencing policy; tutor support 
issues; changes in attitudes and beliefs: literacy use in the community 
(Seminar observation). 
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The 1991 intake participants who were in my sample had all found topics 
which enabled them to satisfy some concern or curiosity about their teaching 
and their students. One teacher, for instance, *had wanted to study at the 
same time the two changes that interested her in her class: in self-attitude and 
in writing. She was studying how the first was reflected in the second' 
(Interview). Participants who had completed projects in earlier years testified 
to their value (though not all had foimd equal value in first and follow-up year 
projects). A typical note in my interview records reads, for instance: *Her 
second year project had been very important to [the interviewee]. It was a 
study of students using micro-computers in groups; she had been concerned to 
fijid what they got out of it' (Interview). For a number of teachers it was 
starting the research project that made AEDP come together to make sense for 
the first time (see below, p.35). 

An example of the supporting documentation was a set of Report Guidelines 
posing questions which research project rqx)rts should address. The questions 
were about the issues that had given rise to the inquiry, the process of 
develops j a research procedure, the means of involving students and fellow 
AEE* participants, piloting, what was learned, and what would be done 
differenfly another time or on a larger scale. 

What participants learned firom the research project appeared to be heavily 
affectai by the quality of their one-on-one communication with the team about 
it. Indeed, a major value of the research project was as a vehicle for effective 
individual teaching. I observed an impressive instance during the individual 
conferences at the follow-up seminar I attended. In the course of the 
consultation one of the team (Deborah) was able to show the participant that 
the project could be a means by which he might address the teaching or 
assessment problems currentiy concerning him: 

*You have questions, then study them in your classroom. F-f^une 
questions like "How does this style of teaching affect my students? 
How does it affect me?". It*s like taping a gig. [The participant had 
been a jazz musician.] You go back to the piano and you say "What 
did I actually do there? Why did that work so well?" Like a science 
experiment - you hold off on your judgement for a couple more 
steps.' 

Deborah showed how to frame a research topic which would allow the teacher 
to study the one student he was particularly intrigued by, at the same time as 
addressing the main area in which his classroom practice needed strengthening, 
that of group management; and finally she got him to see the necessity of 
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exhaustive written recording and the dangers of unrealistically taping more 
material than there would ever be time to analyse (Seminar observation). 



I Some participants saw value beyond their own learning in undertaking the 

research project. 

I 



*[The interviewee] was interested in the issue [portfolio assessment] in 
itself, but also, because the Board of Education was gomg to produce a 
policy of portfolio assessment, she wanted to have an input on this 
before it solidified. She therefore needed to argue in committees and 
felt that doing some research into the issue would give her something 
solid to talk about' (Seminar observation). 

The team's belief was that * writing a research paper had value not because 
competence m the particular written genre was important but as a means of 
acquiring a way of thinldng that would then be taken into teaching' (Interview, 
project team). Not all participants, however, were convinced, *The writing of 
the first year project wasn't an important part of it - doing the project was the 
important thing' (Interview). The general view was, however, that AEDP had 
enabled teachers to realise the power of writing as a means of gaining insight. 

It seemed to be harder for some administrators than for other participants to fit 
the research project model to their situation. One said, for instance, that *she 
experienced the project as something of a distraction from her broad purpose 
of understanding mass placement; she had had to choose a narrower topic for 
the sake of manageability' (Interview). Another had found a satisfying focus 
only for the follow-up year project. 

The heavy investment placed by the team m the research project proved 
generally justified. The project was a valued learning experience that 
many participants believed resulted in permanent gains in skill and 
understanding. It was a vehicle through which AEDP's ideas were 
operationalised and given substance by the teachers within their own 
contexts. The team were especially skillful and effective in supporting the 
projects. 



5.4 Site visits 
According to the team, 

*site visits by team members had not been made in the first year 
[altliough participants had formed research project groups which met in 
their own Jncalities: these had been visited twice]. In the current year 



ERLC 



32 



30 



An Evaluation ofAEDP 



each participant was visited twice and possibly three times. The 
functions of the visit were to get to know the participants, to help them 
with the research project and to provide practical support (such as 
taking notes during the course of an activity). Even though the quantity 
of the support was nominal, isolated teachers found it valuable and the 
journals constantly referred to the helpfulness of the visits. Also, the 
visits bridged the long time gaps between the project's meetings' 
(Interview, project team). 

All the participants who spoke about the visits confirmed that they had found 
them helpful, though one participant in the 1990 intake said that not all 
teachers in her group had welcomed the visits. She herself named feedback 
from Lena about her teaching as particularly helpful. Another said she *had 
had useful ideas from Debbie on assessment, teaching, organization and her 
research project, and that it had been good for h^ students to see someone 
from where she goes when she leaves them on some Fridays' (Interview). 
After a visit from Debbie during which *they had dissected one person's 
portfolio together', a third teacher had been asked to write about the visit in 
her journal and had found the writing ^empowered her and was dynamic' 
(Interview). A fourth had been visited by all three of the team and had gained 
insights which he was able to put to immediate practical use (Interview). 

Participants felt that site visits provided significant help. They are 
particularly useful as a means of helping teachers reinterpret their own 
classrooms in terms of the project's general ideas. 



5.5 Reading 

Copies of one or more articles were given out at the end of most seminars, for 
reading before the next meeting. Participants were expected to write reflective 
journal entries on aspects they found particularly relevant. In the interviews 
specific readings were often mentioned as memorable, useful or illuminating: 
these included chapters from Elsa Auerbach's Making Meaning, Making 
Change and Denny Taylor's and Catherine Dorsey-Gaines's Growing Up 
Literate. Linda Brodky's *The Literacy Letters' was mentioned by several 
participants offering different evaluations: according to one teacher *some had 
been so incensed they had wanted to write to the author, though [this teacher] 
had not herself felt this way' (Interview). An article which spelled out the 
literacy practices engaged in by people in depressed areas of cities, and which 
attacked the misconcq)tion that people in such places don't read, was taken by 
some participants as something of a revelation (Interviews) ^ and by one 
(claiming to express the feeling in the group) as a condescending explanation 
of what any serious adult educator took for granted. This participant had 
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found *the articles early on (in the 1990-91 program) too vague. Last year 
everyone attacked the articles' (Imerview). This was clearly one manifestation 
of the difficulty the team faced in catering for a group which included such a 
range of experience and knowledge. A participant who wanted AEDP to be 
more academic lamaited (alone among my informants) that the readings were 
*truncated and decontextualised; one did not know anything about the authors 
and what else they had done' (Interview). 

The typical reaction to the readings, however, was that they had generally 
been helpful. In a discussion I attended on an article about qualitative data a 
teacher said that reading this had made her notice and make a record of the 
jealous behaviour of her student Doris (see p.24) (Seminar observation). 
Another specific result of an identified reading was one teacher's reported new 
practice of ^monitoring her own talk to avoid judgemental terms. She had 
been made aware of the need for this by a reference in a reading to a student 
who was asked "What helped you most?", and replied "The way you talked"' 
(Interview). Another participant ^intended to use the article ^Collage of 
Portfolios' as a model for writing up her research project' (Interview). 

No set of readings will entirely suit every teacher, especially when 
controversial items are included. The selections worked well to raise 
awareness of issues, stimulate thinking and discussion, and provide 
knowledge of practice elsewhere. 

There would have been no harm, and something to be gained, in the 
team's sharing with participants, through handouts, more of their own 
contextual understandings of the domains from which the articles were 
drawn. Information could, for instance, have been provided about the 
nature of the journals and books the items appeared in, the backgrounds 
and other work of the authors, the range and scope of other articles about 
the topics, and bibliographic sources available. Such a move might have 
been an easy way to help assuage some participants' anxieties about the 
lack of an information base. Serious reading, beyond the sources given, is 
in any case part of the professional culture which projects like AEDP seek 
to create. 



5.6 Journals 

Not all participants fully accepted the need to write prolifically or were 
convinced of its value. One administrator said she got no time to write. 
*There was perhaps too much writing. There needs to be more care given to 
people who do not thrive on writing but who may like simply to talk and 
listen. Those are valid ways of learning' (Interview). Another said that in her 
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first year she had not used the journal as she was meant to, but came to value 
writing in the follow-up year. The team^s position, that writing could make a 
crucial difference to the scope and quality of reflection about teaching, 
seminars and texts, could perh^s only be appreciated by those who were 
prepared to do enough writing of the ^ropriate type to experience these 
benefits. 

Most participants, even if writing had not yet worked this way for them, 
worked hard at their journals in the expectation that the team's beliefs would 
prove well-founded. They either rajoyed writing the journal (*IThe 
interviewee] loved writing the journal and getting re^nses' — Imerview) or 
were persuaded of its value and were disciplining themselves to do it. If the 
journal's reflective function was still not fully appreciated, most had 
discovered its value as a record which enabled one to recall what would 
otherwise have been forgotten, to realise the progress one had made and to 
notice what one had got out of an experience that at the time had seemed 
insignificant. During a seminar I saw teachers reading extracts from their 
journals to the group, a process evidently considered valuable by those 
present. One reader was asked for a copy of a list she had created in one of 
her entries (see below, 5.8). Some participants, though not the majority, 
volunteered a rationale close to that which a member of the team formulated in 
these terms: 

*There is an initial problem for participants of getting clear about what 
all the different elements have to do with each other. Helping to 
overcome this is partly the purpose of the journal' (Imerview, project 
team). 

The rest still saw the journal as primarily an aspect of documentation. 

For reflection and deliberation, not everyone's cognitive style is best 
served by writing. But it could help more teachers than currently know 
how to use it. Moreover, if teachers are to adopt a researcher stance 
towards their work they need techniques of written documentation. The 
team were right to place the emphasis they did on participants' writing 
and were successful in convincing a high proportion of participants of the 
value of writing for their own work. 



5.7 The balance of activities m AEDP 

The combination of meetings, research projects, site visits, reading and 
writing was one of the distinctive contributions of AEDP and worked well, 
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enabling elements to reinforce each other and ideas to be the more 
securely understood through being handled in a number of media and 
contexts. It also helped to ensure that theory and practice informed each 
other. 



5.8 The follow-up year 

There was agreement among the three participants I spoke to from the first 
(1989) intake that the follow-up year had been more rewarding and enjoyable 
than title first year. (Because of iUness one of the three had attended only the 
early follow-up sessions.) A reason cited was that the most argumentative and 
least committed people had dropped out; another was that the group had been 
smaller. A suggestion that, since the reaUy important work was done in the 
second year, a first year January start woidd give sufficient lead-in time, 
presumably reflected that participant's memories of the troubled first few 
months of the 1989 session. This person said that by the follow-up year she 
had learned to use writing quite differently and thus found her involvement 
more productive. She had found her second year research project interesting 
to write and had learned a lot in doing it. Her colleague confirmed that her 
second year project, a study of students using micro-computers in groups, had 
been very important to her, unlike the one she had done in the first year. 

Of the next year's participants (the 1990 intake, constituting the current 
follow-up group) all confirmed the value of the second year, a judgement 
which was not coloured, like that of the first group, by a sense of sharp 
contrast with an unhappy first half-year. The main positively appreciated 
factor was small group size. One of the less confident and experienced 
participants said *she appreciated the smaller group and more relaxed 
atmosphere and did not feel as apprehensive diis year' (Interview). Another 
(who was alone in this) disagreed: *ti[ie best thing about AEDP was the chance 
to dialogue informally with colleagues; tiiie second year group is less 
rewarding because it's smaller' (Interview). Otherwise, the only regret was at 
the infrequency of the meetings, which are held once a month and on two 
Saturdays: *the program loses intensity; [the interviewee] tends not to think 
about the program until a few days before. The frequency of first year 
meetings was fine' (Interview). 

Some members of the 1991 intake expressed regret that there would be no 
follow-up year for them. (Since these interviews it has fortunately proved 
possible after aU to provide a second year program for this group.) 

As in the first year, the seminar was the core of the follow-up year's activity. 
The pattern of the one I observed (on 1 February) was as follows. Participants 
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arrived in the lounge and went into individual or paired consultations with 
team members. These lasted a full hour and a half; the meeting I sat in on 
(between Debbie and an individual participant), and, it appeared, the others 
also, were intense working sessions around the planning and execution of the 
research project (see above, 5.3). The full group then met and exchanged oral 
accounts of their current projects. The teacher working with Debbie told the 
group about useful advice he had just received from her. A great deal of 
discussion was generated. Limch, taken in the lounge, occupied half an hour. 
One participant was invited by the team to read out some reflective writing. It 
was about the difficulty of combining observation with participation and the 
responsibility to be available for help and to keep the class going. Another 
member (as reported above) asked for a copy of this person's list of things that 
she might look for when observing. After a brief discussion of the scheduling 
of future meetings Debbie guided the group in a consideration of an article 
about qualitative data which they had read since last time. The day ended with 
advice on target dates for completing stages of the project, and the handing out 
of readings for the next session. 

Participants who took part in the f oUow-up seminar were unanimous as to 
its value* In part the smaller group size was a factor, but in addition, 
after a full year's work and a completed research project, teachers were at 
this stage in a position to achieve maximum beneHt from working 
together* 



6 The effects of AEDP on participants 



6,1 Participants' experiences of AEDP 

Coming to grips with new ideas. Participants (not only ft'om the first group) 
ft'equently reported a sense of confusion and loss of bearings during the early 
months of their involvement in AEDP. They had found it hard to grasp what 
the project was essentially about. In the earlier (November) set of interviews, 
some of the final (1991) group expressed a sense of disorientation: *[The 
interviewee] was somewhat confused after the first two sessions but now, after 
the third, says she sees where the program is going.* *The initial ideas 
promoted were not too clear to her [a different participant] ... It*s still early 
days and she doesn't feel she's got a handle on things yet' (Interviews). 
Looking back from later in the year and subsequent years, participants tended 
to see their own initial confusion as inevitable given their state of 
understanding at the time; it had not been caused by lack of clarity on the 



37 



An Evaluation ofAEDP 



team's part. *The project, at first, seemed rather directioiiless but this was due 
to the inexperience and lack of understanding of the participants. The team 
were actually telling you things but you didn't understand them' (Interview). 
One teacher spoke of her own and a colleague's experience of the ideas falling 
into place and making sense after the first few sessions: 

*At first, there seemed to be nothing to bite into. The groups were 
used to cookbook approaches and weren't getting them. A turning 
point which she recalled was when [a colleague] suddenly said, in about 
January, "I get it. It's about making assessment part of teaching." 
That was a significant moment. [The interviewee] thought she herself 
had caught on a bit before that' (Interview). 

What helped to bring the project into focus and to provide participants with 
new excitement in learning was the institution of empirical inquiry at their 
sites: *In spring it all got more relevant. It may have been that they had 
moved on to different types of article But also they started gathering data 
themselves and seeing progress in students' (Interview). Others too reported 
on the difference that starting the projects had made. 

Many participants communicated to me, asked or unasked, their 
understandings of the key ideas they had acquired. Both the current year 
participants who provide! comment in April, and members of the earlier 
groups, expressed these ideas in generally consistent terms that were in tune 
with the team's stated intentions. In the participants' formulations some of 
these ideas were: 

encouraging teachers to build in an extensive knowledge of their 
students in such a way as to inform the curriculum. 
A teacher ^should also realize the extensive skills and experiences and 
networks, and the general background knowledge, that students bring 
with them that's usable' (Interview). 

AEDP ^promotes team exploration of ways to help students evaluate 
themselves and to develop non-threatening, encouraging, and useful 
forms of assessment which might eventually displace the more 
disruptive forms of evaluation that are presently used' (Interview). 



Overall evaluation by participants. Specific comments offered by participants 
on AEDP as a whole were as follows: 

*She didn't know when she applied that it was going to be this good. 
It's more than fulfilling expectations' (Interview). 
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*She had no suggestions for improvements' (Interviev^). 
*Feels satisfaction with the understanding she's achieved, though less 
with the p^rwork (project and journal) because she has not found 
time to do them properly' (Interview). 

The project [she was now in the follow-up year] *had been an epiphany 
with the readings and the theory. Things from her life had come 
together in this' (Imerview). 

Implicit in almost all the testimony were a basic approval and appreciation of 
what AEDP was doing. This is equally true of those who were most 
forthcoming with criticisms, and it is against that background that the 
criticisiTiS must be seen. About the team's management style, one participant 
expressed the general view when she said that *the AEDP team were warm, 
encouraging, humorous and down-to-earth, and not threatening; one felt they 
were open to interaction' (Interview). Similarly: *The leaders are dynamic 
and good at research' (Imerview). A third teacher said he *needs a broader 
view and sees it in the way the team work. They operate an effective 
apprenticeship model, and he's drawn to them: the three are very supportive, 
in different ways' (Interview). 

As against the general satisfaction with the project, the criticisms may be 
quickly indicated. All were, as far as I could tell, minority opinions of one or 
a few individuals, and did not represent the feeling in the group as a whole. 
Dissatisfections concerned the first year of AEDP and (often in relation to that 
year) the balance of information-giving with discussion and activity, and of 
more democratic with more directive management of the seminar. One 1989 
participant made a recommendation: *The team should not hold back from 
presenting a considerable amount of information at the start of the program. 
The intention to build up to a collaborative professional relationship is worthy 
but people do need information to work on first. So there shpuld be 
presentations, experts should be brought in and the loosening up should be a 
gradual process' (Interview). As mentioned above (5.2), the view that 
information was what was needed is perhaps open to challenge. 

That the team responded in good time to the problems in the initial year is 
affirmed by more than one participant: 

'After the first meetings, the team sensed the diversity in the group and 
switched to more explicit discussions of what alternative assessment 
meant, whole language, the writirg process and theories of reading. 
They gave a lot of readings in terms of articles. They discussed ways 
of doing research' (Interview). 
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The lack of similar critical comment from members of the 1990 and 1991 
intakes indicates that in those years the team was felt to have got the balance 
right. 

Another respect (not altogether distinct) in which some fault was found with 
the program was the balance of the tiieoretical or academic with the practi.cal. 
The view was expressed (in relation to the 1990-91 program) that more 
'theory' was needed, by which the participant meant that underlying concepts, 
the team's and the participants', should be explicitly formulated for critical 
discussion; this was seen as a means of addressing the diversity of 
assumptions and presuppositions which new groups brought witiii them: 

*There could be more theory and there was not enough time spent 
talking about the theory. The very different assumptions of the 
participants on intake needed to be teased out early so the people 
understood what they were looking at. The project was successful for 
[this informant] because she was already sympsrthetic to the AEDP 
philosophy. For others, the adjustment was difficult* Programs with 
diverse intake are an advantage and so are different ideas but people 
need to be self-conscious about them' (Interview). 

In fact the team believe they included a considerable amount of theory, 
although not through discrete formal presentations. 

An obvious point of comparison with the AEDP program was conventional 
academic courses, and comment was offered on what the relationship should 
be. Not all this comment was consistent. One participant, referring in 
November to a university course she had taken which had been appropriately 
theoretical for the academic context, felt that the AEDP project should in 
comparison remain practically-oriented* The following April, however, she 
was suggesting that * something a bit more like an academic course, with more 
formal teaching — a balance,' would have been better. Another participant, 
however, felt that AEDP would, as it stood, make a good basis for a Masters 
program in Adult Education. 

A criticism voiced in only one interview was that AEDP was unrealistic about 
the practices that AE teachers could be expected to implement. 

*[The interviewee's] main criticism of the project was that things whicli 
are desirable will not be done unless they're realistic. You have to deal 
with the population - with the teachers as they are, not as they should 
be. Her colleagues work thirty hours without preparation time and 
have to do other jobs in order to live. A suggestion of typing up 



40 



38 



An Evaluation ofAEDP 



interviews is therefore quite unrealistic. That's an example. Nobody 
could do fifteen interviews in September and type them up. Even if 
she modelled this sort of conscious behaviour, her colleagues wouldn't 
assimilate it. The team should be asking the question, *What can we 
do that is confined to class time or that won't take extra time'? At 
meetings which she's attended, any ideas that involve more time are 
simply dismissed. One colleague became hysterical at the mere 
mention of portfolios' (Interview). 



Personal development of the participants. Successful professional 
development is often experienced as personal development. This happened in 
AEDP. Sometimes it had to do with the occurrence of AEDP at a particular 
juncture m a person's biography and career, perhaps when feeling stale after 
years in the same job. A general sense of revitalisation could then occur: *the 
project is bringing me back to life' (Interview). 

Some experienced the development of their writing abilities as an enhancement 
tliat went beyond the immediate context of teaching and assessment. One 
teacher spoke of learning through AEDP to organise writing more tightly 
because of the greater awareness she had gained of structure and classification 
(Interview). Another *now does more writing of her own. She likes writing to 
reflect on articles - writing "think-pieces". She also writes now while the 
students are writing. That's a change' (Interview). 



Administrators' experience ofAEDP. Although the team had had doubts 
about the desirability of including administrators in the project, on the grounds 
that their presaice might be inhibiting to teachers, a number had been and 
were involved and I detected no inhibiting effect. One of the administrators 
felt that 'the administrator-teacher mix was actually very good' (Interview), 
but I did not collect teachers' views on the matter. It was one of the 
administrators, working for an organization in which the use of the TABE was 
mandatory, who most obviously illustrated the achievement of one of the 
project's purposes, 'to get the participants to look critically at the practices that 
their programs v^ere telling them to engage in. The project provided a neutral 
ground on which they could look at the official instructions that they were 
having to cope with' (Interview). 

A problem for the project in including administrators was that their needs and 
interests were somewhat different firom those of the teachers. The 
administrator just quoted felt that the conception of the research project in the 
1989-90 year was too teacher-oriented for her needs, though this was rectified 
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in the follow-up year: *She thought that in the second year the team did 
respond adequately to her different situation as an administrator and not a 
teacher' (Interview). An administrator in the current (1991) intake was in no 
doubt that what she had learned from AEDP was directly useful in her work, 
since this was very much about assessment: see below, 6-4, 



Participants were overwhelmingly approving of AEDP. They enjoyed their 
involvmient and thought they learned a great deal from it. They 
appreciated the team's seriousness, expertise and respect for what the 
teachers brought with than, and the dunate of open interaction which was 
created. Some had conspicuously acquired new professional skills, beyond 
the usual classroom ones, such as writing and systematic inquiry. 

Adnnnistrators' needs were somewhat diflTerent but it seemed that these 
could be successfully accommodated within a group basically consisting of 
teachers, and that there were advantages in their participation. 

In relation to participants' overall approval of AEDP, criticisms were 
minor. Some dissatisfactions concerned the amount of what was variously 
refen^ to as 'information' and ^theory'. But participants were confused 
on this point, perhaps indicating a need for some explicit discussion of 
what 'theory' is: the respects, for instance, in which it can be given and 
those in which it can be grasped only through active engagement with 
problems, and the fact that it may be manifested not only in formal 
exposition but in the treatment of substantive issues and experiences. 



6.2 Effects on teachers' practice 

The most obvious indicator of the effectiveness of AEDP would be changes in 
participants' practice. A limitation of an evaluation that depends mainly on 
interviews with participants and examination of documents is that it gains little 
direct evidence of changes in practice. Indirect evidence, however, is in this 
case available and useful. It comes most frequently from interviews, but also 
from journals and research projects. 

The apparent simplicity of the criterion of ^changed practice* in evaluating a 
project such as AEDP is, however, misleading. For instance, procedures 
advocated by the project might indeed have been implemented, but in a 
mechanical manner; for an adaptive use of AEDP ideas within the changing 
situations of teaching, an understanding of principle is required.. What is at 
issue is thus not simply changes introduced but improved capability to make 
appropriate change. A related consideration is tliat procedures endorsed by 
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AEDP might already be being practised by a teacher, but simply as self- 
devised or imsystematically acquired pragmatic solutions to immediate 
problems. The same practices are likely to be deployed more effectively, 
confidently and adaptively (and to be disseminated to colleagues) if they are 
grounded in a woriid-out system of ideas. Thus one participant, according to 
the team, was referred to as ^having been given permission by the project to 
go on and do the things that she had been doing instinctively. It had confirmed 
her intuitive knowledge and made it more Intimate' (Interview, project team). 
(Such cases are, however, sometimes difficult to distinguish fix)m those in 
which participants look for confirmation for their own habitual practices and 
fail to note ways in which these are contradicted by the principles established 
in AEDP. Such may have been the case with the teacher who said *it was 
good to have endorsement as a good approach for sometk'iig that you've been 
doing for a long time perh^s instinctively' — Interview.) 

A number of teachers when asked were able to list the ways in which their 
thinking and practice had changed: 

*A closer attention to the individiial, sharpening of self'<x)nsciousness, 
developing the discipline of writing things down, encouragement of 
reflectiveness' (Interview). 

^Noticing what students are and aren't doing; realising the importance 
of student involvement and ownership; realising the value of 
discussions, and awareness of the need to develop skills of managing 
discussion; and realising that with support most individuals are able to 
work out their problems: the task is to open the way for them' 
(Interview). 

*More open to students; less formal and reserved (students now use 
her first name); more alert to what students say: picks up on it; 
monitors her own talk to avoid judgemental terms; more observant of 
the way students behave; more mvolved in who they are as 
individuals; uses students as a resource; involves them in self- 
assessment; mdividualised approach to spelling and vocabulary; 
generally, has cleared up in her mind how to get students involved' 
(Interview). 



Understanding and adoption of alternative assessment. Perhaps the most 
important understanding gained in the area of assessment was that the process 
could not be precise or mechanical: 
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* Since experts do not have answers (even in the diagnosis of 
physiologically-based problems), trial and error aiq)roaches are 
necessary. The building of sensitivity and awareness is appropriate 
training in assessment, so that teachers become better at explaining and 
more sensitive to issues and clues. Training of teachers therefore can 
only be through fostering awareness and sensitivity, and cannot be 
through learning 10 things to do, a how-to manual' (Interview). 

This teacher had learned to see her own notemaking in class as an assessment 
activity. Part of what was implied in that view was *a diff^nt relationship 
with the students in which they and the teacher are co-investigators and co- 
planners'. The documentation and analysis become a shared attempt to 
understand. 

Another participant had learned consciously to value, in the context of 
assessment, what she learned when *her students from Cuba and the Caribbean 
teU her all about a building they remember, the school they went to and so on. 
This is all very helpful.' It seems to have been AEDP that introduced her to 
the possibility of systematically seeking such data through interviews - a 
practice which, as an adininistrator, she is able to introduce across her site 
(Interview). 

It is possible, however, that some participants incorporated interviewing and 
forms of self-evaluation simply as technical strategies, without understanding 
or accepting the basic idea of teacher and student together arriving at an 
adequate rounded picture. A possible indicator of this failure was the worry 
expressed by one 1991 intake teacher early in the program about possible 
*bias' introduced by the interview process and the difficulty of obtaining 
^objective' information: *How do you measure the results of an interview?' 
(Interview). 

A number of participants had instituted procedures whereby students could 
assess their own progress. Some sounded rather routine in nature, such as a 
* sheet which she requires the students to complete daily in which they say how 
well they've done, what they've done, how they feel - just a sentence - and 
also for each reading selection, students make an assessment on interest, main 
idea and comment' (Interview). This teacher's later (April) report of what she 
had been doing, however, suggested that her approach to self-assessment had 
been more radical: she claimed she had been 

*able to move from a whole-class to an individualised mode of learning 
in math because the students had been put in charge of monitoring and 
recording their own progress. As a result students feel more 
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accountable and less dependent, and [the teacher] believes this has 
contributed to their current sense of achievement in this subject' 
(Interview). 

At a local level self-evaluation might simply be a matter of ©nicouraging 
students' explicit awareness of their own strategies: one teacher asks, for 
instance, *How do you know that word?' (Interview). 

Teaching: more specific ejects. One teacher insisted that she still 
maintained certain practices which she felt that the team disapproved of: 
^recognizing phonic patterns is helpful.' In fact, the team believe this too but 
want participants to have a sense of the relative importance of this skill in the 
whole of reading practice. This teacher went on, however, to acknowledge 
that *she also uses techniques offered here such as reading journals and 
journals in general' (Interview). A second teacher, speaking of an exercise she 
had used before (writing a biogr^hical sketch), explained that *This time she 
has approached it differently, and attributes the difference explicitly to what 
she has learned from AEDP. She has talked more with the students, observed 
more, documented more, and given more help. The work has been more 
individualised and more relevant' (Interview). 

Practice related to writing had been affected, according to the testimony, in a 
number of ways. The dialogue journal had, according to the testimony, been 
adopted in a number of classes. A teacher from the current group had this 
year *used more writing than ever before' (Interview). Another, who had been 
made very conscious of the value of writing for students and had made it a 
central part of her program, was pleased that after some months students were 
developing individual styles of writing and were making choices that were less 
dependent on her suggestions. The students were now comfortable writing 
essays and were moving towards the target of the essay required in the GED 
test (Interview). 

Teachers as researchers. The team were convinced of the power of the 
process of * systematically gathering, analysing and using information on their 
students. One team member hesitated to use the term **scientific stance*" 
because of misleading associations, but that conveyed the two elements of 
deliberateness and reflection which were consider^ essential' (Interview, 
projea team). Only careful documentation over a period would allow patterns 
to be revealed. A researcher's approach to teaching would enable practitioners 
to go beyond what intuition, sensitivity and common sense could bring on their 
own. ^Research' for the team was not simply an honorific term for more alert 
and attentive teaching but implied the positive adoption of distinct systematic 
procedures. 
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An qpisode in one of the follow-up seminars provided an instance of the value 
which the team and some participants found in systematic observation and 
documentation. Addressing one teacher's expressed bafflement and confusion 
at the failure of her students to make api»rent progress, one of the others put 
it to her that 

^''You know, the question of whether there is something specific 
holding them back is something that we can't answer without data. It 
would be useful to be able to say, *What is holding this person back is 
so and so\" One of the team added that close observation of the 
students could help instructionally also; she cited examples' (Seminar 
observation). 

A participant in the current intake group mentioned an awareness she had 
developed as a result of practising the systematic observation and recording 
she had learned in AEDP: she was comparing the reading strategies of four 
students, and *said she wouldn't have had the knowledge to make this 
comparison without AEDP. Training ^labled you to see more, notice things: 
AEDP shapes awareness' (Interview). Another teacher 

^appreciated the concept of ongoing documentation in evaluation and 
assessment and acknowledged that we don't make time to write it 
down: she tended to make a note in her head, which was not so good 
because she was bound to forget it in two or three years time. 
Documentation was vital for convincing agencies on issues related to 
individuals: '*This person did this ... When I interviewed her again I 
realised ... "' (Interview). 

A third teacher was finding a new intellectual pleasure in employing the 
research process as a means of illumination. She was detennined to 

*keep fuller journals in future, to document and reflect so as to draw 
better conclusions and compare teaching methods. She didn't document 
before, or analyse properly. She now liked the research aspect more 
than any other, and particularly the coding process for its objectivity, 
its removal of personal feelings and prejudices' (Interview). 

Implicit in the last point may perhaps be a lurking delusion that objectivity free 
of observer's values is attainable through more systematic research; elsewhere 
this participant showed a persistent belief in the need for quantifiable 
measures. Nevertheless, there is no doubt from her testimony that the 
procedures she had adopted had led to greater knowledge of her students and 
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improved insights into ways of releasing their energies and confidence as 
learners. 

AEDP saw the solution to the teacher element in the general problem of 
quality in adult education as essentially a matter of creating a profession, or at 
least a professional culture, constituents of which would be professional 
education and a research tradition to gen^te and constantly revitalise the 
understandings teachers needed. Research which would make a difference 
needed to be done by teachers. The way the team viewed research was as the 
systematic and planned end of a continuum, at the other end of which was 
habitual reflection in the course of teaching. AEDP was particularly effective, 
as was generally acknowledged, in putting into the hands of the practitioners 
the tools they needed for research. It also developed skills, habits and 
experience in a process the team saw as essential at all points of the teaching- 
research continuum, that of writing to record, reflect and analyse. Several 
participants who had not seen themselves as writers had learned to do so, and 
some who already made a practice of writing found their sense of its 
possibilities extended. (Others, as we saw, never fully took to the journal 
form as a means of thinking, maintaining it principally as a simple record - 
though they acknowledged the benefits of returning to the record to observe 
their own development over a period of months.) 

Evidence suggested that a good proportion of the teachers had changed 
and were chang ing their practice m teaching and assessment as a result of 
AEDP. The project's success in this was inevitably uneven given the well- 
known difficulties of changing teachers' entrenched assumptions, and in 
the face of the unfavourable working situations of some teachers. The 
skiUs and techniques necessary for changed practice had certainly been 
imparted to a good number of participants. 



6.3 Success and failure in AEDP: contributmg factors 

AEDP was clearly more successful with some participants than others and it is 
worth looking for patterns in this. Cases in which less had been gained from 
the experience sometimes seemed explicable in terms of the particular 
participants* situations: inexperience coupled with isolation, for instance, 
together with lack of pre-service training, little help on the job and students 
with low motivation and very basic literacy abilities. Such a combination of 
adverse circumstances could overwhelm a teacher's confidence in her ability to 
improve her students' situation, and undermine her expectations that research 
activities would provide information which might make a difference to her 
effectiveness as a teacher. ITie way the more successful AEDP participants 
constantly reported new discoveries about their students' resources and new 
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teaching approaches which had opened up possibilities, was in striking 
contrast. 

The rarity of relative failure, however, shows by contrast how effective AEDP 
was in general in convincing participants of the value of its approaches and 
getting them to the point of experiencing benefit from their application. The 
participant who was probably the most sceptical of those I spoke to had 
nevertheless been eager to return for the follow-up year, currently put energy 
into her writing and research, and acknowledged that she derived great 
satisfaction from the new skills she had developed. 

Logically, a concern with assessment issues is secondary to a concern with 
teaching and learning. Making assessment the focus of AEDP assumed 
participants who had a conceptual grasp of, and experience in, the curriculum 
and pedagogy of adult literacy/ESOL education. It would seem plausible that 
inexperience in adult education might preclude fruitful engagement with 
AEDP. Numerous cases, however, contradict that conclusion. Two teachers 
stood out as having limited experience and yet clearly benefiting greatly from 
the process. Both, however, had teaching experience in other sectors 
(elementary and high school), and had pursued studies, formally and 
informally, in education; issues of teaching and learning bad been continuing 
concerns. The critical criterion, then, might not be experience of teaching 
adult literacy/ESOL so much as a history of engagement, conceptual and 
practical, with educational problems. 



6.4 Impact on programs 

The aspirations of AEDP were not simply to influence the group of 
practitioners who participated in the project each year but to have some effect 
on the system more widely. Although the form of the evaluation made it 
unlikely that adequate evidence would be collected on this, some indications 
are nevertheless worth reporting. 

One obvious channel of influence was through the administrator members of 
the project. The administrator whom I interviewed from the current (1991) 
group was very clear: 

*IThe interviewee] had joined AEDP out of an administrative concern 
with assessment for initial placement. The alternative assessment 
techniques learned on AEDP had been implemented and had made a 
difference. The standardized test had been supplemented by altemative 
assessment: this had already resulted in some students being placed in 
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Level 3 instead of Level 1 (as the test had indicated), with striking 
effects on the students' confidence. The teachers are happy that they 
now know more about the studrats before meeting them: they have the 
record of the interview and some writing. The resulting documentation 
is useful politically: the supervisor now trusts [the interviewee's] 
opinion more, so that she was able (for instance) successfully to argue 
for an increase of the number of staff involved in the initial placement 
process' (Interview). 

Teacher participants had initiated new assessment practices alongside the 
existing tests where those were used- 

*When [the interviewee's] students left her after a few months to go to 
the Occupational Education teachers, she now sent with them portfolio 
samples and evaluation sheets as well as the TABE scores, so that the 
latter could be critically evaluated: the teachers taking over the 
students might note, for instance, that despite the scores "This person 
can assimilate and organise data"' (Interview). 

Teachers and administrators could also extend the impact of AEDP by relaying 
its message to others in their programs. There is evidence that this had 
happened and would continue to happen. One of the participants whose 
responsibilities included staff development said he *brings the AEDP materials 
into his weekly staff development sessions and talks about them;' while a 
second, who had no opportunity to offer formal training, operationalised her 
understanding through introducing alternative assessment procedures and 
instructing her team in their use. Of the regular teachers, one had planned her 
research project so as to gather evidence with which to influence an area of 
policy-making in her program (see above, 5,3); and another had, with a 
colleague who was also an AEDP participant, already conducted workshops 
for teachers across her community organization: 

*They have done two presentations on alternative assessment, 
addressing the issue of whether there is some alternative to the 
standardized test, one for other teachers in the same employment- 
oriented program, the other for teachers in [the community 
organization], 18 teachers in all. They exposed these groups to their 
own classroom activities, through display and discussion: observation, 
interviews, portfolios, dialogue journals. Both audiences were 
recqptive. [The interviewee] and her colleague argued that although 
they could not replace the standardized test teachers should document 
and keep records, so that when students don't perform well on tests 
there's an alternative means to prove what they can do. Teachers saw 
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the value of the ^roach, and have started to attempt to implement it, 
seeking further advice from the two of them' (Imerview). 

A third teacher had serious intetitions of offering workshops on alternative 
assessment, first at her own site and then at Board of Education Professional 
Development meetings: *DocumCTiting and analysing her own classroom work 
will be her research project; the report will be evidence to present to other 
teachers that alternative assessment tools work' (Imerview). 

Amongst the more distant aims of the team in setting up AEDP was to create 
more public and professional awar^ess and debate about assessment, and to 
increase pressure for change in the standard testing arrangements for adult 
literacy/ESOL education in the city. There is no knowing how far AEDP has 
in feet contributed to such developments, though it seems likely that through 
its very identification of assessment as a topic meriting two years of educators' 
systematic attention, and its high profile (not least with the main agencies, who 
indicated their agreement by sending participants) it will have reinforced the 
growing perception of assessment as a critical issue that wiU have to be 
tackled. 

Some administrators were introducing changes in assessment procedures 
because of AEDP, and some teachers were actively disseniinating their new 
expertise to others in their programs and organizationso It was too early 
to judge how effective this process was. 



7 AEDP, assessment, and improving adult education 
Beyond the local institutional concerns and opportunities that immediately gave 
rise to the project, might AEDP's approaches form a possible basis for the 
more general professional development of the community of adult educators? 

In the final set of five interviews I received three usable answers to the 
question, *How much do you feel AEDP could supply towards a fuUy adequate 
professional training, if such were planned?' One said the whole course 
should be included, though did not say what else would be needed. Another, 
noting that AEDP was appropriate only for experienced teachers who knew the 
ills of the standardized test, named the topics which would need to be added: 
adults as learners distinct from children and how to teach them; the 
importance of self-concept for adult learners; multicultural awarenesses; 
teaching through issues like AIDS and healthcare; the situations of adult 
learners (needs, welfare dependence, concerns); and specific approaches to 
reading, writing and mathematics. A third considered that AEDP constituted 
the essential basis of a Masters in AE (which he wished Lehman College 
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would offer); it had enabled him to look at his own teaching and had given 
him tools he needed, such as analysis; AEDP's teaching of ethnographic 
research methods was at Masters or Doctoral leveL 

One of the features most 2^)preciated about AEDP by some participants was 
that it had brought them from their marginal position in the education system 
into contact with the mainstream, by illuminating thdr own specialism with 
concepts and readings drawn from central strands in current educational 
thinking. Some participants saw others within the group as rq[)res^ting that 
mainstream. A teachCT who worked in what she saw as the peripheral 
specialism of occupational education felt that 'In g^eral it was good that 
having been very much a detail person herself, somebody who wrote detailed 
curriculums concerned with mechanics, the project was enabling her to 
broaden out, making her look at broader issues. Other teachers on the project, 
she had noted, spoke in broad professional terms. This was the first time that 
her group, occupational education teachers, had participated in other than basic 
training* (Interview). 

It is worth noting incidentally one specific sense in which AEDP would 
provide an excellent base on which to ground future projects and curricula: 
AEDP conspicuously practised what it preached in the area of documentation. 
The team were able to hand me a complete (and very fat) dossier reflecting the 
project^s transactions since its inception. This included agendas and briefing 
notes which the team had constructed for themselves in preparation for 
seminars, and which were annotated with notes made for future reference 
during or after the event, recording how things went and what might have 
been done differently. In addition, from the second year Deborah Shelton 
produced detailed ^minutes* of the seminars - including verbatim citation of 
exchanges - for distribution to the participants. Along with these documents, 
readings and handouts were carefully preserved. Not only did this make the 
evaluator^s job especially easy (I was able, for instance, to 'triangulate' one 
participant's interview account of her AEDP experience with both her journal 
and the minutes of the seminars to which she contributed), but a successor 
project would be able to set out where AEDP left off and have the benefit of 
its thinking, mistakes and successes; even the team's tacit knowledge, so 
easily lost when any enterprise comes to an end, is to some extent preserved in 
the commentaries and memos which these understandings implicitly informed. 

Assessment as the focus. The focus of AEDP was not on the total 
professional development needs of literacy/ESOL teachers but specifically on 
one issue, assessment. I wondered at first whether this apparently instrumental 
focus was merely a strategy to seduce hard-headed teachers into thinking more 
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broadly about the education they were offering. This proved not to be so: 
AEDP really was about assessment. The team were happy when participants 
were led by new assessment thinking to reconsider aspects of their pedagogy 
also, but spin-offs for teaching were seen as just that. (I recorded instances of 
the team exploiting teachers* fresh ideas about teaching to make them aware of 
related possibilities for assessment.) 

On the oth&c hand, in the AEDP perspective assessment in significant ways 
was teaching; this was one crucial differrace between alternative and 
traditional assessment. The surprise for the hypothetical hard-bitten 
practitioner who joined the project would be, not that assessment was just a 
cover for the real issue of curriculum and pedagogy, but that assessment was 
something quite different from what he or she, and the devisers of the familiar 
tests, had always supposed. Assessmrat was knowing the students. That 
included, as in the test, knowing what they could do, but knowing it in an 
elaborated and discriminating way that distinguished what they could do in 
different circumstances. It also included knowing why the students could do 
what they did; what the ways of thinking, feeling and operating were that had 
brought them to, or stopped them at, their current stage of capability; and 
what they might be expected to become able to do with support. 

The adoption of this extended meaning of assessment was not an attempt, 
through semantic trickery, to switch the agenda back to student-centered 
education. Rather it was to take seriously the functions which the standardized 
tests were intended to perform, and to find better ways of fulfilling them. The 
TABE and GED tests were used for placement within and movement between 
levels and sectors of education and the employment market, and to measure 
the effectiveness of programs; if the insistent testimony of team and 
participants is to be believed, the tests did not do the job well. Students were 
regularly mislabelled and misplaced in ways that harmed both them and the 
classes, programs and jobs they were placed in; important progress went 
unrecognised, while teaching which produced an ability to pass tests but not to 
function in everyday circumstances served nobody's real needs, except perhaps 
those of the bureaucracy. Conventional assessment, moreover, left 
unaddressed a further urgent need, that of providing information on which 
teaching decisions could be made. 

Adequate data for such judgements could be obtained only by other means, 
notably the collection and recording of more diverse and extensive 
manifestations of the student's thinking and performance, including, where 
possible, in the process of education. It is this more extended and ambitious 
process that is referred to as alternative assessment. 
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It is the requirement to gather information on stud^ts in the course of their 
education that has far-reaching implications for teaching- Alternative 
assessment involves careful observation and noting of what students do and 
say, and analysis of their writtoi work. But as well as carrying out their 
normal teaching functions with more of an eye and an ear for what is going on 
in their classes, teachers practising such assessment cannot help changing their 
teaching behaviour also- They find themselves interacting more with students, 
eliciting responses and behaviour in order to gather information or test 
hypotheses, inviting studaits to evaluate themselves, and sharing insights with 
students about them and about teaching and learning- The net effect of these 
assessment-led processes is to change the character of the classroom, a change 
to which numerous participants testified in detail. Performances elicited under 
the name of alternative assessment (for instance, in journals, self-assessment 
reports and interviews) become significant parts of tiie education process and 
help to extend students' c^bilities. Education becomes a more ftiendly, 
supportive, responsive and respectful process, something which students to a 
greater extent do for themselves rather than which is done to them- 

In short, although the overt attempt in this project was not to tackle the whole 
job of improving the quality of adult education, it in fact made a significant 
inroad into that broader agenda. If that was what one wanted to do through a 
program of in-service professional development, there would be worse places 
to start than with assessment. 

AEDP has pioneered a curriculum and pedagogy which in its content and 
its participative and mteractive approach would have much to contribute 
to any general program of khservice professional development for teachers 
of adult literacy/ESOL* The issue of assessment was not only an 
important and effective focus in its own right but proved a highly 
appropriate vehicle for more general professional development. 

8 Conclusions and recommendations 
Summary of main evaluation points 

The seminars. In general the seminars were highly valued by participants and 
met several of their most urgent needs: for meeting colleagues from 
different sites, for sustained examination of their practice and for 
exposure to and communal reflection on ideas from sources outside the 
city (particularly through articles)- Allowing teachers to meet on a 
regular basis for extended discussion, organized and informal, was 
perhaps what participants valued the most about AEDP. 
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The informality and collegiality of the management of the seminars 
made them enjoyable and minimised for less experienced participants 
any anxieties about contributing to discussion in a large group. Though 
the ^proach was at times difficult to handle, it is hard to imagine how 
a more prescriptive or tightly scheduled approach could have achieved 
the same ends. 

There may be room, however, for some tactical concessions to 
conventional expectations in the early weeks with a group. The team 
were right in principle to resist pressure to collude with some 
participants' demands for a heavy loading of information. Conscious 
thought does, however, need to be given to overcoming participants' 
early anxieties about whether their investment of time and energy will 
be worthwhile. Helpful strategies might include demonstrating at the 
start that significant work exists in alternative assessment and that 
participants would become femiliar with it, and, second, creating a 
sense of businesslike briskness in the proceedings (p.26). 

The research project. The heavy investment placed by the team in the 

research project proved generally justified. The project was a valued 
learning experience that many participants believed resulted in 
permanent gains in skill and understanding. It was a vehicle through 
which AEDP's ideas were operationalised and given substance by the 
teachers within their own contexts. The team were especially skillful 
and effective in supporting the projects (p.29). 

Site visits. Participants felt that site visits provided significant help. They are 
particularly useful as a means of helping teachers reinterpret their own 
classrooms in terms of the project's general ideas (p.30). 

Readings. No set of readings wiQ entirely suit every teacher, especially when 
controversial items are included. The selections worked well to raise 
awareness of issues, stimulate thinking and discussion, and provide 
knowledge of practice elsewhere. 

There would have been no harm, and something to be gained, in the 
team's sharing with participants, through handouts, more of their own 
contextual understandings of the domains ft*om which the articles were 
drawn. Information could, for instance, have been provided about the 
nature of the journals and books the items appeared in, the backgrounds 
and other work of the authors, the range and scope of other articles 
about the topics, and bibliographic sources available. Such a move 
might have been an easy way to help assuage some participants' 
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anxieties about the lack of an infonnation base. Serious reading, 
beyond the sources givra, is in any case part of the professional culture 
which projects like AEDP seek to create (p.31). 

Writing. For reflection and deliberation, not everyone's cognitive style is best 
served by writing. But it could help more teachers than currently know 
how to use it. Moreover, if teachers are to adopt a researcher stance 
towards their work they need techniques of written documentation. The 
team were right to place the emphasis they did on participants' writing 
and were successful in convincing a high proportion of participants of 
the value of writing for their own work (p,32). 

TTie balance of activities. The combination of meetings, research projects, 
site visits, reading and writing was one of the distinctive contributions 
of AEDP and worked well, ^labling elements to reinforce each other 
and ideas to be the more securely understood through being handled in 
a number of media and contexts. It also helped to ensure that theory 
and practice informed each other (p.33). 

Follow-up year. Participants who took part in the follow-up seminar were 
unanimous as to its value. In part the smaller group size was a factor, 
but in addition, after a Ml year's work and a completed research 
project, teachers were at this stage in a position to achieve maximum 
benefit from working together (p.34). 

Participants' experience ofAEDP. Participants were overwhelmingly 

approving of AEDP. They enjoyed their involvement and thought they 
learned a great deal from it. They appreciated the team's seriousness, 
expertise and respect for what the teachers brought with them, and the 
climate of open interaction which was created. Some had 
conspicuously acquired new professional skills, beyond the usual 
classroom ones, such as writing and systematic inquiry. 

Administrators' needs were somewhat different but it seemed that these 
could be successfully accommodated within a group basically consisting 
of teachers, and that there were advantages in their participation. 

In relation to participants' overall approval of AEDP, criticisms were 
minor. Some dissatisfactions concerned the amount of what was 
variously referred to as information' and ^theory'. But participants 
were confused on this point, perhaps indicating a need for some explicit 
discussion of what ^theory' is: the respects, for instance, in which it 
can be given and those in which it can be grasped only through active 
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engagement with problems, and the fact that it may be manifested not 
only in formal exposition but in the treatment of substantive issues and 
experiences (p.39). 

Changes in practice. Evidence suggested that a good proportion of the 

teachers had changed and were changing their practice in teaching and 
assessment as a result of AEDP. The project's success in this was 
inevitably uneven given the well-known difficulties of changing 
teachers' entrenched assumptions, and in the fece of the unfavourable 
working situations of some teachers. The skills and techniques 
necessary for changed practice had certainly been imparted to a good 
number of participants (p.44). 

Effects on programs. Some administrators were introducing changes in 
assessment procedures because of AEDP, and some teachers were actively 
disseminating their new expertise to others in their programs and 
organizations. It was too early to judge how effective this process was (p.47). 

AEDP and professional development. AEDP has pioneered a curriculum and 
pedagogy which in its content and its participative and interactive 
approach would have much to contribute to any general program of in- 
service professional development for teachers of adult literacy/ESOL. 
The issue of assessment was not only an important and effective focus 
in its own right but proved a highly appropriate vehicle for more 
general professional development ^.50). 



Recommendations 

1 AEDP's effective combination of activities combined with the focus on 
assessment is worth repeating and expanding. 

2 There is a need, more generally, for a professional context in which 
teachers of adult literacy/ESOL from different institutions and programs 
can meet regularly, minimally to share ideas and experiences and 
ideally to build a common knowledge culture through systematic joint 
inquiry. 

3 Additional resources available to a future project could be valuably 
deployed on regular site visits, which greatly enhance the effectiveness 
of the training. 
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4 With bimonthly meetings two years are needed for teachers to gain the 
full benefit from a program like AEDP. Organizations which restrict 
teachers' release to one year may risk losing their investment because 
changes in thinking and practice are not securely enough established. 

5 Thought needs to be given to the relationship between an AEDP type of 
program and more academic studies. Providers need to be clear - and 
explicit with participants - about such issues as what theory means in 
the current project, what knowledge and information could usefully be 
presented as such, whether any part of the aim is to lead some teachers 
toward more academic study, and the respects in which the standards 
and criteria normally applied to research for, say, a master's degree are 
appropriate or in^propriate for the studies teachers undertake in 
relation to thdr own practice. A project's view on these matters would 
affect, for instance, the way readings and references were presented, 
whether and how participants were introduced to bibiographic sources, 
and the rigor with which claims in their papers were critiqued. 

6 A future project might include funding to provide continuing support 
for teachers and administrators undertaking to disseminate new 
approaches through workshops in thdr organizations. 
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APPENDIX 

Construction of the Sample of Participants 

The strategy was agreed with the team that participants from the first two 
intakes would be interviewed once and from the current intake twice. The 
number of participants it would be possible to interview during visits to the 
city was judged to be about twelve. Selection criteria were agreed with the 
team to yield representation of all the catteries they considered significant. 
These were, for the two earlier years: the degree of benefit which in the 
team's judgement they had derived from participation in AEDP, and 
participation in the follow-up year; and for all years: experienced or 
inexperienced in their current area of teaching; type of students taught 
(ESOL, ABE, begimiCT, advanced); and type of organization (of the four 
targeted in the design of AEDP). A list of all participants for all three years, 
marked with these categories and with additional written comments, was 
provided by the team. I made a long-list selection from these by a mechanical 
process, indicating first choices and reserves. Substitutions were later made, 
by agreement with the team, for selected participants who could not be 
contacted for interview, were unavailable or whose sites were so remote that 
the time spent in visiting them would have severely reduced the number of 
interviews which could be conducted. (A number of participants were in the 
event interviewed at Lehman College while they were attending the seminar.) 
The difficulties in these respects proved considerable, so that the resulting 
sample, particularly of the 1989 group, was less balanced than might have 
been hoped. 

The characteristics of the sample of 11 as finally agreed and interviewed were 
as follows: 

From the 1989 participants, 3 were interviewed; from 1990, 4 (the whole of 
the current follow-up group with the exception of one auditor); from 1991, 4. 

Participants varied in how useful the team estimated AEDP had been to them, 
though most were thought to have derived substantial benefit; they tended to 
confirm these estimations. Participants thought to have learned little were both 
fewer in numbers and more likely to have lost contact with the project, less 
readily reachable for interview, and less willing to give their time to assist the 
evaluation. For similar reasons participants who had not joined the follow-up 
groups were not able to be included. 

In the other categories (level of experience, type of student, type of 
organization, the sample was well mixed. 
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